1. Field of the Invention
The present invention relates to an image processing apparatus and a method of processing an image, and more particularly, to an image processing apparatus and a method of processing an image to reduce noise of the image.
2. Description of the Related Art
As an image noise reduction method with an edge preservation effect, Lee's method (see J. S. LEE, “Digital enhancement and noise filtering by use of local statistics”, IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-2, pp. 165-168, 1980) has been known. In this method, a variance of signal levels of pixels of an image in which pixels are arrayed in XY directions is calculated, and noise reduction is performed based on the variance, as expressed in Equations (1) and (2).r(x,y)=(1−p(x,y))·E(g(x,y))+(px,y)·g(x,y)  (1)
                              p          ⁡                      (                          x              ,              y                        )                          =                                            σ              g              2                        ⁡                          (                              x                ,                y                            )                                                                          σ                g                2                            ⁡                              (                                  x                  ,                  y                                )                                      +                                          σ                n                2                            ⁡                              (                                  x                  ,                  y                                )                                                                        (        2        )            
In Equation (1), g(x,y) denotes the signal level of a pixel positioned at coordinates (x,y) of an image. E(g(x,y)) denotes a mean value of signal levels of pixels in a local area centering on the coordinates (x,y). In particular, E(g(x,y)) is a mean value of signal levels of pixels positioned within a range of 5 pixels×5 pixels which centers on the pixel of the coordinates (x,y). In Equation (2), σ2g(x,y) denotes a variance of signal levels of pixels in a local area which centers on the pixel of the coordinates (x,y). In particular, σ2g(x,y) denotes a variance of signal levels of pixels positioned within a range of 5 pixels×5 pixels which centers on the pixel of the coordinates (x,y). σ2n(x,y) denotes a variance of a noise level included in a signal level of a pixel positioned within a range of 5 pixels×5 pixels centering on a pixel of coordinates (x,y).
An edge feature amount p(x,y) denotes an edge degree of the pixel positioned at the coordinates (x,y). The feature amount p(x,y) has a value ranging from 0 to 1 inclusive. As apparent from Equation (2), as the noise variance value σ2n(x,y) which is previously measured increases, the feature amount p(x,y) decreases, and as the local variance value σ2g(x,y) increases, the feature amount p(x,y) increases. Therefore, as can be seen from Equation (1), as the noise variance σ2n(x,y) increases, a ratio which the local mean value E(g(x,y)) occupies in the signal level r(x,y) after noise reduction to the original signal level g(x,y) increases. On the other hand, as the local variance σ2g(x,y) increases, a ratio which the local mean value E(g(x,y)) occupies in the signal level r(x,y) after noise reduction to the original signal level g(x,y) decreases.
In an edge part of an image, the local variance value σ2g(x,y) of an image increases. Therefore, in an edge part of an image, the value of the edge feature amount p(x,y) approaches “1”, and a ratio which the original signal level g(x,y) occupies in the signal level r(x,y) after noise reduction increases. As a result, in an edge part of an image, it is possible to prevent an edge of the original signal level from being blunt by the local mean value E(g(x,y)).
Meanwhile, an image in which a noise of an impulse form such as a bright spot noise, a dark spot noise, or a random telegraph signal (RTS) noise is present due to a defective pixel has the large local variance value σ2g(x,y) and so is likely to be determined as an edge part. Therefore, at positions in which the noises are generated, the value of the feature amount p(x,y) increases, and a noise reduction effect is reduced.
FIG. 23 illustrates a relationship between the edge feature amount p(x,y) and a standard deviation σg(x,y) of an image. FIG. 23 illustrates an example when the standard deviation σn(x,y) of a noise is 250 in an image of a 14-bit pixel, in which the horizontal axis denotes the standard deviation σn(x,y) of a noise, and the vertical axis denotes the edge feature amount p(x,y).
A median filter is commonly used to remove an impulse noise. However, when the median filter is used, a high frequency part of an image is distorted, so that detail information of an image is lost. It is known that a photoelectric conversion element used as a digital camera image sensor causes a low frequency noise such as a thermal noise or a 1/f noise which occurs in an amplifying circuit as well as a dark current noise and an optical shot noise which occur at the time of photoelectric conversion (for example, see H. Tian, B. Fowler, and A. E. Gamal, “Analysis of temporal noise in cmos photodiode active pixel sensor”, IEEE J. Solid State Circuits, vol. 36. Jan. pp. 92-101, 2001).
Of these noises, in particular, values of a dark current noise and an optical shot noise depend on a signal level, and thus a variance value σ2n(x,y) of the noises also depends on a signal level. It is known that with the recent miniaturization of a pixel, in particular, in a complementary metal oxide semiconductor (CMOS) image sensor, a low frequency noise called a random telegraph signal (RTS) noise generated by a reading transistor of a pixel causes a noise of a white spot form in an image (for example, J. Y. Kim, et al., “Characterization and Improvement of Random Noise in 1/3.2” UXGA CMOS Image Sensor with 2.8 um Pixel Using 0.13 um-technology”, IEEE Workshop on CCD and AIS, p. 149, 2005).
In the case of a noise of a high level or a noise which is spatially expanded like a thermal noise of a digital camera, it is effective to perform noise reduction processing as in Equations (1) and (2) described above using a large size kernel (referred to as M×N pixels). However, when a kernel size is large, there is a problem in that a calculation amount increases. To cope with this problem, a technique of using an image which has a plurality of different resolutions (for example, see Japanese Patent No. 3193806 or Japanese Patent No. 3995854) has been suggested.
It is well known that an image file format generated by a digital camera is greatly divided into two. One is a universal file format such as Joint Photographic Experts Group (JPEG) or Tagged Image File Format (TIFF), which is recorded after so-called development processing is performed within a digital camera. The other is a device-dependent file format called RAW format, in which an output of an image sensor is recorded as is.
The RAW format has an advantage of being capable of obtaining a result of changing various parameters (such as an exposure value and a white balance value) without deteriorating an image quality (for example, see Japanese Patent Application Laid-Open No. 2004-128809).
In recent years, a digital camera has a trend that as the number of pixels of an image sensor increases, a pixel is miniaturized, and as the sensitivity of an image sensor increases, a noise caused by the sensor increases. In a digital camera, since a noise is generated for various causes, image processing on a signal in which various noise characteristics are reflected is performed. As a pixel is miniaturized, a noise of a pulse form is generated. To remove a noise of an impulse form through processing using Equations (1) and (2) described above, it is necessary to increase a noise removal level. However, when a noise removal level increases, there is a problem in that a high frequency component of an image may be lost, and an edge may become blurred.